Latent Class Analysis

• Latent Class Analysis (LCA) is a probabilistic method of unsupervised clustering that can be utilized when it is believed that there may be unobserved subgroups (classes) among the individuals within a population. (Nylund-Gibson and Choi 2018)

• The central assumption of LCA model is the presence of latent classes within a population. (Weller, Bowen, and Faubert 2020)

• LCA is a type of finite mixture model (FMM) which is a statistical approach used in unsupervised learning.(Grimm, Houpt, and Rodgers 2021)

• LCA is a useful model for identifying subgroups within a population based on patterns in categorical variables.

• In LCA, the variables are unrelated within each subgroup but similar across each one.

Advantages:

• Identifying hidden subgroups

• Handling categorical data

• Providing probability estimates

• Handling missing data

• Model selection

Drawbacks:

• LCA is computationally demanding, limiting the number of variables in the analysis

• It may be challenging to determine appropriate number of latent classes.

The standard equation (Naldi and Cazzaniga 2020) for LCA model is:

\[ p(x_i)=\sum_{k = 1}^{K}{p_k}\prod_{n = 1}^{N}{p_n(x_{in}|k)},\]

• \(p(x_i)\): the probability of observing a particular combination of responses in a group of \(N\) variables

• \(p_k\): the probability of membership in LC \(k\)

• \(p_n(x_{in}|k)\): the probability of response to variable \(n\), conditional on membership in LC \(k\)

Two parameters of an LCA model:

• Inclusion probabilities (\(p_k\))

• Conditional probabilities (\(p_n(x_{in}|k)\))

The general steps to use latent class analysis are as follows:

• Identify the research question and define the variables

• Select the appropriate software

• Determine the number of latent classes for an ideal model

• Estimate the model: Estimate the model parameters using maximum likelihood estimation or Bayesian estimation

• Evaluate the model: Goodness-of-fit statistics (the likelihood ratio test, Akaike Information Criterion \((AIC)\), Bayesian Information Criterion \((BIC)\), entropy and chi-square Goodness-of-Fit \((\chi^2)\))

• Conduct sensitivity analysis: Finally, conduct sensitivity analyses (e.g. correlation test) to test the robustness of the results and evaluate the stability of the estimated probabilities across different subgroups or samples.

To assess the effectiveness of Latent Class Analysis, we used the ‘Zoo Animals’ dataset from Kaggle. This dataset has 101 entries and is an ideal choice because of its majority of binary variables.

To perform our analysis, we are using these 14 variables.

In order to determine the optimal number of classes that would best model our dataset, we conducted a series of analyses beginning with 2 classes and concluding at 7 classes, which was the number of different types of animals in the dataset. The code presented below was utilized to fit the model for each of the different number of classes.

library(poLCA)

lca_fit2 <- poLCA(lca_bind, data = new_zoo_int_1, 
                  nclass = 2, graphs = FALSE, na.rm = TRUE, 
                  nrep=100, maxiter=100, verbose = FALSE)

The poLCA function has the following options: -nclass -graphs -na.rm -nrep -maxiter -verbose

This function is estimating “Latent Class Prevalence”, which is the probability that each dataset entry belongs within one of the model-generated classes (Law and Harrington 2016)

Entropy is a measure of the concentration in a probability function(R Core Team 2021)

We use the poLCA.entropy() function to compare our 5-Class and 6-Class models.

library(poLCA)

lca_fit5.ent <- poLCA.entropy(lca_fit5)
lca_fit6.ent <- poLCA.entropy(lca_fit6)

Class 1 which, as the largest class, is estimated to contain ~35.46% of the population predicts its members to have ‘hair’, ‘toothed’, ‘backbone’, ‘breathes’, ‘tail’, ‘has_legs’. We filtered the cleaned new_zoo dataset for these attributes, and noted that all members were classified as type ‘mammal’ in the original dataset.

Class 2 which is estimated to contain ~20.78% of the population predicts its members to have ‘feathers’, ‘eggs’, ‘backbone’, ‘breathes’, ‘tail’, ‘has_legs’. We filtered the cleaned new_zoo dataset for these attributes, and noted that all members were classified as type ‘bird’ in the original dataset.

Class 3 is estimated to contain ~17.82% of the population. It should be noted that it has two attributes (‘breathes’ and ‘predator’) which were both over 50% shown on the graph, but as we excluded our initial filtering assessment to ~80% and above, these are not selected in the filter.

These class members are predicted to have ‘eggs’ and ‘has_legs’. While this limited selection duplicates a few of the previous members that were selected for other classes, this class also contains the invertebrates and insects, which were not present in the other classes. Based solely upon outside comparison to the type attribute in the original dataset, this class appears to be the least homogeneous.

Class 4 which is estimated to contain ~13.09% of the population is less homogeneous than Classes 1 & 2, but not as heterogeneous as Class 3. The original classification schema of “type” had 7 unique values, which implies that with a 5-class model, overlap is inevitable.

This is only visible due to the fact that we have the original data for “type” for illustration purposes, which is unlikely to be the case in research data.This class’ members are predicted to have ‘aquatic’, ‘predator’, ‘toothed’, ‘backbone’, ‘breathes’, ‘tail’. We filtered the cleaned new_zoo dataset for these attributes. The resulting filter only yielded 5 members, which are mostly aquatic mammals and a single amphibian.

Class 5 which is the smallest class, estimated to contain ~12.85% of the population predicts its members to have ‘eggs’, ‘aquatic’, ‘toothed’, ‘backbone’, ‘fins’, ‘tail’. We filtered the cleaned new_zoo dataset for these attributes, and noted that all members were classified as type ‘fish’ in the original dataset.

As seen in the correlation heatmap figure below, ‘milk’ is highly positively correlated with both ‘hair’ and negatively correlated with ‘egg’. We decided to remove ‘milk’ as an attribute, and re-consider our LCA model to see if it improves without the variables exhibiting this high degree of multicollinearity included.

Given improvement in all fit statistics except entropy for the 5-Class model with the high correlation variable ‘milk’ removed, we have decided to remove ‘milk’ for our final model and are left with 13-attributes for optimal classification by Latent Class Analysis.

Class 3 & Class 4 show the greatest adjustments to the population proportions as compared to the 5-class model with ‘milk’ still considered. Class 3 sees about a 1% drop in membership, and Class 4 sees ~1.4% increase. Because these two classes were our most homogeneous, it makes sense that they might see the greatest changes in proportion.